docs: consolidate to user-facing guide only (removes duplicates and legacy flat docs)#1029
docs: consolidate to user-facing guide only (removes duplicates and legacy flat docs)#1029FlorianBruniaux wants to merge 10 commits intodevelopfrom
Conversation
fix(refacto-codebase): technical docs & sub folders
…-components--rtk chore(master): release 0.34.2
fix(refacto): wrappers for standardization, exit codes lexer tokenizer, constants, code clean
…-components--rtk chore(master): release 0.34.3
Creates docs/README.md (interface contract defining 3-tab structure, required frontmatter, and conventions) and 4 stub pages with valid frontmatter for docs/guide/, docs/reference/, and docs/architecture/. These are the prerequisites for the prepare-docs.mjs pipeline (Plan B) and for Adrien's Phase 1 content work. No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> Signed-off-by: Florian BRUNIAUX <[email protected]>
Creates the complete user-facing guide for the docs website: Getting started: - quick-start.md: 5-minute walkthrough (init, first commands, rtk gain) - supported-agents.md: Claude Code, Cursor, Copilot, Gemini, Cline, Windsurf, Codex, OpenCode — integration tiers, install commands, graceful degradation Commands (adapted from FEATURES.md, English, --help-first format): - git.md: status/log/diff/show/add/commit/push/pull/branch + gh CLI - cargo.md: test/nextest/build/check/clippy/install + generic test/err wrappers - files.md: ls/read/grep/find/diff/wc/smart with before/after examples - javascript.md: vitest/playwright/tsc/eslint/prettier/next/pnpm/npm/npx/prisma - python.md: pytest/ruff/mypy/pip/deps Reference: - filters/using-filters.md: 8-stage pipeline, lookup priority, TOML DSL reference, Mermaid diagram (adapted from docs/filter-workflow.md) - analytics/gain.md: rtk gain flags, daily/weekly/monthly breakdowns, export formats, token estimation, database management (from docs/AUDIT_GUIDE.md) - configuration.md: full config.toml reference, env vars, tee system, telemetry - troubleshooting.md: common issues and fixes (from docs/TROUBLESHOOTING.md) All pages carry valid frontmatter (title, description, sidebar.order). No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> Signed-off-by: Florian BRUNIAUX <[email protected]>
…mmands Guide — 10 new command and analytics pages: - commands/go.md: go test (NDJSON), golangci-lint - commands/ruby.md: rspec, rubocop, rake - commands/dotnet.md: dotnet build/test, binlog, format - commands/containers.md: docker ps/images/logs/compose, kubectl pods/services/logs - commands/github-cli.md: gh pr/issue/run, Graphite gt commands - commands/data.md: json, env, log, curl, wget, aws, psql, summary - commands/utilities.md: proxy passthrough, global flags, RTK_DISABLED - filters/creating-filters.md: TOML DSL guide, field reference, inline tests - analytics/discover.md: rtk discover — missed savings analysis - analytics/economics.md: rtk cc-economics — dollar savings vs ccusage Reference tab (9 pages): - contributing/guide.md: design philosophy, PR process, TOML vs Rust decision - contributing/security.md: vulnerability reporting, dangerous patterns, dep criteria - contributing/coding-standards.md: Rust rules, error handling, lazy_static, fallback pattern - contributing/testing.md: snapshot tests, token accuracy, cross-platform, benchmarks - internals/command-routing.md: 6-phase lifecycle, exit codes, verbosity, module map - internals/filter-pipeline.md: 12 filtering strategies, TOML stages, savings by ecosystem - internals/tracking-system.md: SQLite schema, data flow, token estimation, Tracker API - internals/hook-engine.md: rewrite registry, compound commands, exit code contract, rtk init - toml-dsl/specification.md: complete field reference, pipeline order, build compilation Architecture tab (4 pages): - diagrams/command-flow.md: end-to-end Mermaid — hook → RTK → LLM - diagrams/filter-pipeline.md: build pipeline + 8-stage runtime Mermaid - decisions/why-no-async.md: ADR — single-threaded, <10ms startup constraint - decisions/proxy-architecture.md: ADR — why CLI proxy over aliases/LD_PRELOAD/hooks-only All pages carry valid frontmatter (title, description, sidebar.order). No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> Signed-off-by: Florian BRUNIAUX <[email protected]>
|
|
i18n docs — two options, need inputBefore this merges, worth aligning on how we handle multi-language docs down the road. The current structure is EN-only (explicit decision in Plan B), but the question is whether we set it up to be migration-friendly. Option A — locale subdirectory at docs/ root
Upside: clean, native Starlight i18n, no URL changes for EN. Option B — EN-only in repo, translations externalized Translations live on Crowdin/Weblate or a separate repo. Upside: zero impact on current structure, zero maintenance burden until there's actual demand. My take: Option B now, Option A when a specific language has contributors ready to maintain it. Migration is a one-liner at that point. No reason to restructure prematurely. @pszymkowiak @aeppling — what's your preference? If you want Option A, better to decide before Adrien starts filling in content so we're not doing a mass rename mid-sprint. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new structured documentation layout under docs/ intended for consumption by the rtk-ai/rtk-website pipeline (Guide / Reference / Architecture), and also includes release/version metadata updates.
Changes:
- Add a large set of new docs pages with required frontmatter, organized into
docs/guide/,docs/reference/, anddocs/architecture/. - Add diagrams (Mermaid) and ADR-style architecture decision pages.
- Update release/version files (
CHANGELOG.md,Cargo.toml,Cargo.lock,.release-please-manifest.json).
Reviewed changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 20 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/README.md | Defines the 3-tab docs structure and frontmatter/linking contract. |
| docs/guide/index.md | Guide landing page linking to key user-facing sections. |
| docs/guide/getting-started/installation.md | Installation + verification instructions. |
| docs/guide/getting-started/quick-start.md | 5-minute walkthrough for initial usage. |
| docs/guide/getting-started/supported-agents.md | Lists supported agents and integration tiers. |
| docs/guide/configuration.md | User-facing config and env var reference. |
| docs/guide/troubleshooting.md | Common issues and fixes. |
| docs/guide/filters/using-filters.md | Explains TOML filters, lookup order, and pipeline (contains schema details). |
| docs/guide/filters/creating-filters.md | Walkthrough for writing project-local TOML filters. |
| docs/guide/commands/git.md | Git command behavior and savings examples. |
| docs/guide/commands/cargo.md | Cargo command behavior and wrappers. |
| docs/guide/commands/files.md | Filesystem/search command behaviors. |
| docs/guide/commands/javascript.md | JS/TS ecosystem commands and examples. |
| docs/guide/commands/python.md | Python ecosystem commands and examples. |
| docs/guide/commands/go.md | Go ecosystem commands and examples. |
| docs/guide/commands/ruby.md | Ruby ecosystem commands and examples. |
| docs/guide/commands/dotnet.md | .NET ecosystem commands and examples. |
| docs/guide/commands/containers.md | Docker/Kubernetes command behaviors. |
| docs/guide/commands/data.md | JSON/env/log/curl/wget/aws/psql/summary commands. |
| docs/guide/commands/github-cli.md | gh/gt behaviors and examples. |
| docs/guide/commands/utilities.md | Proxy passthrough + global flags. |
| docs/guide/analytics/gain.md | rtk gain usage and data model explanation. |
| docs/guide/analytics/discover.md | rtk discover usage and output explanation. |
| docs/guide/analytics/economics.md | rtk cc-economics usage explanation. |
| docs/reference/index.md | Reference tab landing page. |
| docs/reference/contributing/guide.md | Contributor guide distilled for the docs site. |
| docs/reference/contributing/security.md | Security policy + dangerous patterns guidance. |
| docs/reference/contributing/coding-standards.md | RTK-specific Rust coding standards. |
| docs/reference/contributing/testing.md | Testing approaches (insta, savings checks, perf). |
| docs/reference/internals/command-routing.md | High-level routing lifecycle explanation. |
| docs/reference/internals/filter-pipeline.md | Overview of filter strategies + TOML pipeline. |
| docs/reference/internals/tracking-system.md | SQLite tracking system explanation (schema/paths/etc.). |
| docs/reference/internals/hook-engine.md | Hook/rewrite architecture and contracts. |
| docs/reference/toml-dsl/specification.md | TOML DSL “specification” reference page. |
| docs/architecture/index.md | Architecture tab landing page. |
| docs/architecture/diagrams/command-flow.md | Mermaid diagram for end-to-end command flow. |
| docs/architecture/diagrams/filter-pipeline.md | Mermaid diagram for TOML filter build/runtime pipeline. |
| docs/architecture/decisions/why-no-async.md | ADR documenting “no async runtime” rationale. |
| docs/architecture/decisions/proxy-architecture.md | ADR documenting the CLI proxy pattern decision. |
| CHANGELOG.md | Adds release entries for 0.34.2 and 0.34.3. |
| Cargo.toml | Bumps crate version to 0.34.3. |
| Cargo.lock | Updates locked package version to 0.34.3. |
| .release-please-manifest.json | Updates release-please tracked version to 0.34.3. |
docs/guide/filters/using-filters.md
Outdated
| [[filters]] | ||
| name = "my-tool" | ||
| match_command = "^my-tool\\b" | ||
|
|
||
| strip_lines = [ |
There was a problem hiding this comment.
The TOML example uses [[filters]] with a name field and strip_lines, but RTK's TOML engine expects schema_version = 1 at the top and filter definitions under [filters.<filter-name>] with fields like strip_lines_matching / keep_lines_matching (see src/core/toml_filter.rs). As written, this example will fail to parse and the filter will be ignored.
| [[filters]] | |
| name = "my-tool" | |
| match_command = "^my-tool\\b" | |
| strip_lines = [ | |
| schema_version = 1 | |
| [filters.my-tool] | |
| match_command = "^my-tool\\b" | |
| strip_lines_matching = [ |
| S1["1. strip_ansi"] --> S2 | ||
| S2["2. replace"] --> S3 | ||
| S3{"3. match_output\nshort-circuit?"} | ||
| S3 -->|"✅ match"| MSG[["emit on_match\nstop"]] |
There was a problem hiding this comment.
In the runtime pipeline diagram, match_output is shown emitting on_match, but the TOML schema uses match_output = [{ pattern, message, unless? }] and there is no on_match field (see src/core/toml_filter.rs). The diagram should reference the rule’s message instead.
| S3 -->|"✅ match"| MSG[["emit on_match\nstop"]] | |
| S3 -->|"✅ match"| MSG[["emit rule message\nstop"]] |
| **Storage locations:** | ||
| - Linux: `~/.local/share/rtk/tracking.db` | ||
| - macOS: `~/Library/Application Support/rtk/tracking.db` | ||
| - Windows: `%APPDATA%\rtk\tracking.db` | ||
|
|
There was a problem hiding this comment.
The DB filename/location here doesn’t match the current implementation: the tracker uses dirs::data_local_dir()/rtk/history.db by default (not tracking.db), and on Windows it’s under LocalAppData (not %APPDATA%). Please update the paths to match src/core/tracking.rs (HISTORY_DB = "history.db").
| ``` | ||
| estimate_tokens(text) = text.len() / 4 | ||
| ``` | ||
|
|
||
| ~4 characters per token average. Accuracy: ±10% vs actual LLM tokenization. |
There was a problem hiding this comment.
Token estimation is described as text.len() / 4, but the implementation uses ceil(chars/4) (so 1–4 chars still counts as 1 token) in estimate_tokens (src/core/tracking.rs). Please update the formula here to avoid confusion when validating numbers.
| [package] | ||
| name = "rtk" | ||
| version = "0.34.1" | ||
| version = "0.34.3" | ||
| edition = "2021" |
There was a problem hiding this comment.
PR description says “No existing files were modified” and “All changes are additive”, but this PR changes the crate version (and also updates CHANGELOG/Cargo.lock/release manifest). If the version bump is intentional, please update the PR description; otherwise, consider dropping these release/version changes from this docs-only PR.
| [tracking] | ||
| enabled = true # enable/disable token tracking | ||
| history_days = 90 # retention in days (auto-cleanup) | ||
| database_path = "/custom/path/tracking.db" # optional override | ||
|
|
There was a problem hiding this comment.
Docs imply tracking.history_days controls retention, but Tracker::cleanup_old() uses DEFAULT_HISTORY_DAYS and doesn’t read config (src/core/tracking.rs). Consider clarifying this (or implementing config-driven retention) so the config docs match runtime behavior.
|
|
||
| | Variable | Description | | ||
| |----------|-------------| | ||
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | |
There was a problem hiding this comment.
The env var table is missing RTK_DB_PATH, which overrides the tracking DB location with highest priority (src/core/tracking.rs). Adding it here would make configuration docs complete.
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | | |
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | | |
| | `RTK_DB_PATH` | Override the tracking database path | |
| ELSE 0 END | ||
| ), | ||
| exec_time_ms INTEGER | ||
| ); |
There was a problem hiding this comment.
This schema omits columns that RTK adds via migrations (exec_time_ms and project_path) and also has timestamp populated by RTK (not a DEFAULT). Please update the schema snippet to reflect the actual CREATE TABLE + migrations in src/core/tracking.rs so readers can query the DB correctly.
| ## Thread safety | ||
|
|
||
| Single-threaded execution with `Mutex<Option<Tracker>>` for future-proofing. No multi-threading currently used. |
There was a problem hiding this comment.
The “Thread safety” note mentions Mutex<Option<Tracker>>, but the current implementation doesn’t use that pattern (it opens connections via Tracker::new() and relies on WAL + busy_timeout for concurrent access). Please update this section to match src/core/tracking.rs.
| | `keep_lines_matching` | regex[] | no | Keep only lines matching at least one pattern | | ||
| | `replace` | array | no | Regex substitutions: `{ pattern, replacement }` | | ||
| | `match_output` | array | no | Short-circuit rules: `{ pattern, message }` | | ||
| | `truncate_lines_at` | int | no | Truncate lines longer than N characters | |
There was a problem hiding this comment.
match_output rules also support an optional unless regex to prevent short-circuiting when errors/warnings are present (see MatchOutputRule in src/core/toml_filter.rs). Documenting unless here would make the spec complete.
|
Hey, Thanks for the documentation rework, since PR #869 we already had the technical documentation , this is maintained in each respective folder per responsability, to ensure easy collaboration and work in specific areas. We need a clean separation between user documentation : Usage, How to use, How to configure, and so on.... And the technical documentation: How to contribute, Flow Diagrams, Edge case, Engine This is to keep things simple for users, we do not want noise in those documentation. Current RTK technical doc implementationThe documentation follows a distributed, co-located pattern - each folder contains a README explaining its purpose: Root Level Documentation https://github.com/rtk-ai/rtk/blob/main/docs/contributing/ARCHITECTURE.md (42K) Components level Documentation https://github.com/rtk-ai/rtk/blob/main/src/core/README.md Ecosystem-specific READMEs: https://github.com/rtk-ai/rtk/tree/main/src/cmds/git - Git ecosystem filters Full reviewCore Problems
|
|
Thanks for the detailed review — the points are valid, taking them seriously. What you're right about The The commands pages showing What this PR should actually be User-facing only, minimal scope:
For the "what does RTK optimize" question: a single page listing ecosystems and savings metrics (no Proposed next steps
Does this align with what you had in mind? Happy to rework the PR scope before Adrien starts on content. @adrien-epling — also curious what you'd actually want as a user landing on the docs for the first time. |
…flat docs - Remove docs/reference/ (10 files): duplicates CONTRIBUTING.md, SECURITY.md, ARCHITECTURE.md, src/*/README.md (distributed pattern from PR #869) - Remove docs/architecture/ (5 files): duplicates ARCHITECTURE.md ADRs and diagrams - Remove docs/guide/commands/ (12 files): showed rtk <cmd> syntax users never type (hooks rewrite transparently); content was derived from FEATURES.md - Remove docs/guide/filters/ (2 files): contributor content, belongs with src/filters/ - Remove docs/guide/analytics/discover.md, economics.md: implementation details - Remove legacy flat docs/ (6 files): FEATURES.md (FR), TECHNICAL.md, TROUBLESHOOTING.md, AUDIT_GUIDE.md, tracking.md, filter-workflow.md — all superseded by guide/ rewrites or codebase distributed docs - Add docs/guide/what-rtk-covers.md: replaces 12 command pages with one user-facing overview of 60+ commands by ecosystem, no rtk <cmd> syntax - Enrich docs/guide/analytics/gain.md: absorb advanced workflows from AUDIT_GUIDE.md (CI integration, pandas analysis, cron snapshots) - Adapt docs/guide/getting-started/quick-start.md: remove rtk <cmd> examples, explain hook transparency instead - Update docs/README.md: single-tab contract, pointer to codebase for technical docs - Update docs/guide/index.md: clean navigation matching final structure - Add Mermaid diagrams to src/filters/README.md: build pipeline + filter lookup (preserves content from filter-workflow.md in its natural home) Result: 9 files in docs/guide/ (was 39), zero legacy flat docs, zero duplicates. Codebase distributed docs (ARCHITECTURE.md, CONTRIBUTING.md, src/*/README.md) remain the source of truth for technical and contributor content. Signed-off-by: Florian BRUNIAUX <[email protected]>
FlorianBruniaux
changed the title
|
Hey Patrick, thanks for the thorough review — you were right on both counts. What changed in this revision: The original PR had two fundamental problems: Going deeper into the audit, the 6 legacy flat docs ( Result: 9 files instead of 39, all user-facing. Where the removed content lives:
The codebase distributed docs stay as the source of truth for contributor and technical content. |
What
Restructures
docs/to contain only user-facing website content, following the distributed documentation pattern from PR #869.Before: 39 files across 3 tabs (guide/reference/architecture) — duplicates of codebase docs + command pages showing
rtk <cmd>syntax users never type.After: 9 files in a single
docs/guide/tab. Zero legacy flat docs. Zero duplicates.Why
Two root problems identified in review:
reference/andarchitecture/duplicatedARCHITECTURE.md,CONTRIBUTING.md,src/*/README.md(distributed, co-located pattern from PR fix(refacto-codebase): technical docs & sub folders #869)rtk git status,rtk cargo testetc. — syntax users never type because hooks rewrite transparentlyGoing deeper: the 6 legacy flat docs (
FEATURES.md,TECHNICAL.mdetc.) already duplicated the codebase. The 39 new files were a 3rd layer on top of that. This PR collapses all three into one clean structure.Final structure
Where the removed content lives
docs/reference/contributing/CONTRIBUTING.md,SECURITY.mddocs/reference/internals/ARCHITECTURE.md,src/*/README.mddocs/architecture/ARCHITECTURE.mdADR sectiondocs/guide/commands/(12 pages)what-rtk-covers.md(reframed without rtk prefix)docs/FEATURES.md(FR, 1400 lines)what-rtk-covers.md(EN)docs/TECHNICAL.mdARCHITECTURE.md+src/*/README.mddocs/TROUBLESHOOTING.mdguide/troubleshooting.mddocs/AUDIT_GUIDE.mdguide/analytics/gain.md(enriched with CI/cron workflows)docs/tracking.md(Rust API)src/core/README.md(already covers schema + contracts)docs/filter-workflow.md(Mermaid)src/filters/README.md(diagrams added in this PR)Test plan
cargo test: 1253 passed, 0 failuresrtk git/rtk cargo/rtk tscetc. indocs/guide/(onlyrtk gain,rtk init,rtk verify,rtk proxywhich are user-typed commands)src/filters/README.mdnow includes build pipeline + filter lookup Mermaid diagramsGenerated with Claude Code